Solid-state imaging device

ABSTRACT

A solid-state imaging device according to an embodiment includes: an imaging element including an imaging area formed with a plurality of pixel blocks each including pixels; a first optical system forming an image of an object on an imaging surface; and a second optical system re-forming the image, which has been formed on the imaging surface, on the pixel blocks corresponding to microlenses, the second optical system including a microlens array formed with the microlenses provided in accordance with the pixel blocks. The microlenses are arranged in such a manner that an angle θ between a straight line connecting center points of adjacent microlenses and one of a row direction and a column direction in which the pixels are aligned is expressed as follows: θ&gt;sin −1 ( 2 dp/D ml ), where D ml  represents microlens pitch, and dp represents pixel pitch.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromprior Japanese Patent Application No. 2012-249321 filed on Nov. 13, 2012in Japan, the entire contents of which are incorporated herein byreference.

FIELD

Embodiments described herein relate generally to solid-state imagingdevices.

BACKGROUND

In recent years, CMOS image sensors have been actively developed.Particularly, as miniaturization (according to a design rule forminiaturization) is progressing in semiconductor processes, single-panelcolor image sensors with more than 10 million pixels at pixel pitch of1.4 μm, for example, are already available on the market. In the trendof the increasing numbers of pixels, attempts are being made to obtainphysical information that has not been obtained due to the use of alarge number of pixels, such as a distance to an object.

There has been a known imaging element that obtains information aboutthe distance from the imaging element to an object by inserting amicrolens array as a compound-eye optical system between the imaginglens and the imaging sensor. However, with this imaging element, the S/Nratio and the resolution of an image cannot be restored satisfactorilywhen the single image is re-formed by combining images formed byrespective microlenses.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1( a) and 1(b) are diagrams showing a solid-state imaging deviceaccording to an embodiment;

FIG. 2 is a diagram showing a microlens array that is a hexagonalclosest packed array used in the solid-state imaging device of theembodiment;

FIG. 3 is a diagram showing a microlens array that is a square arrayused in the solid-state imaging device of the embodiment;

FIG. 4 is a diagram for explaining image overlaps that appear when theimaging magnification N is 0.5;

FIG. 5 is a diagram for explaining image overlaps and added portions;

FIG. 6 is a diagram for explaining the condition for two overlappingpixels to exist in the same row;

FIG. 7 is a diagram for explaining angles θ that should be avoided in ahexagonal closest packed array;

FIG. 8 is a diagram for explaining angles θ that should be avoided in asquare array;

FIG. 9 is a diagram showing example angles θ that should be avoided in ahexagonal closest packed array;

FIG. 10 is a diagram showing example angles θ that should be avoided ina square array; and

FIG. 11 is a block diagram showing a solid-state imaging deviceaccording to another embodiment.

DETAILED DESCRIPTION

A solid-state imaging device according to an embodiment includes: animaging element including an imaging area formed with a plurality ofpixel blocks each including a plurality of pixels; a first opticalsystem configured to form an image of an object on an imaging surface;and a second optical system configured to re-form the image, which hasbeen formed on the imaging surface, on the plurality of pixel blockscorresponding to a plurality of microlenses, the second optical systemincluding a microlens array formed with the plurality of microlensesprovided in accordance with the plurality of pixel blocks. The pluralityof microlenses are arranged in such a manner that an angle θ between astraight line connecting center points of adjacent microlenses and oneof a row direction and a column direction in which the pixels arealigned is expressed as follows:θ>sin⁻¹(2dp/D _(ml))

where D_(ml) represents microlens pitch, and dp represents pixel pitch.

The following is a detailed description of embodiments, with referenceto the accompanying drawings.

FIG. 1( a) shows a solid-state imaging device according to anembodiment. The solid-state imaging device of this embodiment includesan imaging optical system (an imaging lens) 10, an optical systemincluding a microlens array 20 having microlenses arranged in an array,and an imaging element 30. The imaging element 30 includes an imagingarea having pixel blocks each including pixels. Each of the pixel blockshas pixels arranged in the X-axis direction and the Y-axis direction inan array. The imaging optical system 10 forms an image of an object 100on a virtual imaging surface 70. The optical system including themicrolens array 20 has microlenses provided in accordance with the abovementioned pixel blocks, and re-forms the image, which has been formed onthe above described virtual imaging surface 70, on the pixel blockscorresponding to the respective microlenses.

FIG. 1( b) shows the shift lengths of image points of the same object inthe optical system (a virtual image optical system) of the solid-stateimaging device according to this embodiment.

When attention is paid only to the imaging optical system (the imaginglens 10), the principal ray and its family of rays from the object 100form an image on the virtual imaging surface 70 determined by the focallength of the imaging optical system 10 and the distance from the object100, so as to satisfy the relationship expressed by the equation (1).

$\begin{matrix}{\frac{1}{f} = {\frac{1}{A} + \frac{1}{B}}} & (1)\end{matrix}$

Here, f represents the focal length of the imaging lens 10, A representsthe distance from the object-side principal surface of the imaging lens10 to the object 100, and B represents the distance from the image-sideprincipal surface of the imaging lens 10 to the virtual imaging surface70. The magnification (the lateral magnification) of the imaging lens 12is expressed by the following equation (2).

$\begin{matrix}{M = \frac{B}{A}} & (2)\end{matrix}$

In this embodiment, the virtual imaging surface 70 of the imaging lens10 is located behind the imaging element 30 (or on the opposite side ofthe imaging lens 10 from the object 100). At this point, the microlensarray (hereinafter also referred to as ML) 20 is located in front of thevirtual imaging surface 70, and therefore, light is collected on thesurface of the imaging element 30 that has pixels arranged in front ofthe virtual imaging surface 70 in practice. As a result, the light raysform a reduced image compared with the virtual image. The microlensimaging system forming the microlens array 20 is expressed by thefollowing equation (3).

$\begin{matrix}{\frac{1}{g} = {{- \frac{1}{C}} + \frac{1}{D}}} & (3)\end{matrix}$

Here, g represents the focal length of the microlenses, C represents thedistance from the object-side principal surface of the microlenses tothe virtual imaging surface 70, and D represents the distance from theimage-side principal surface of the microlenses to the surface of theimaging element 30. At this point, the magnification of the microlensimaging system is expressed by the following equation (4).

$\begin{matrix}{N = \frac{D}{C}} & (4)\end{matrix}$

Here, the variable E shown in the equation (5) in terms of a geometricrelationship is introduced. When the optical system is a fixed focusoptical system, the variable E is a fixed set value.E=B−C  (5)

Here, D_(ML) represents the alignment pitch of microlenses 14 or thedistance between the microlenses in a case where two adjacentmicrolenses are selected. At this point, light rays 84 a, 84 b, 84 c,and 86 emitted from the same object form images at the adjacentmicrolenses 14 independently of one another. The distance D_(ML) and theimage shift length Δ on one side are expressed by the equation (6) interms of the geometric relationship among the principal rays 84 a, 84 b,and 84 c at the respective microlenses shown in FIG. 1( b).

$\begin{matrix}{\frac{C}{D_{ML}} = \frac{D}{\Delta}} & (6)\end{matrix}$

With the above parameters, the changes in the respective parameters withrespect to movements (changes in A) of the object 100 are expressed.Where A₀ represents the distance from the imaging lens 10 to the objectthat can be imaged, each parameter having a subscript 0 to the rightthereof (B₀, C₀, D₀, and Δ₀) represents a value that is achieved whenthe distance to the object is A₀. When A₀ is determined, each of theabove mentioned parameters is uniquely determined in a fixed focusoptical system.

Here, M (imaging lens magnification) represents the change in theparameter D that is observed when the distance from the imaging lens 10to the object 100 changes from A₀ to A. According to the equations (1)through (5), the imaging lens magnification M has the relationshipexpressed in the equation (7).

$\begin{matrix}{D = {\left( {\frac{1}{g} + \frac{1}{C}} \right)^{- 1} = {\left( {\frac{1}{g} + \frac{1}{\left( {B - E} \right)}} \right)^{- 1} = {\left( {\frac{1}{g} + \frac{1}{B - \left( {b_{0} - C_{0}} \right)}} \right)^{- 1} = \left( {\frac{1}{g} - \frac{1}{{- \frac{D_{0}}{N_{0}}} + {\left( {M_{0} - M} \right)f}}} \right)^{- 1}}}}} & (7)\end{matrix}$

Also, according to the equations (1), (2), (6), and (7), the distance Afrom the imaging lens 10 to the object 100, the shift length Δ, and themagnification M have the relationship expressed in the followingequation (8).

$\begin{matrix}{A = {\left( {\frac{1}{f} - \frac{1}{B}} \right)^{- 1} = {\left( {\frac{1}{f} - \frac{1}{B_{0} - C_{0} + C}} \right)^{- 1} = \left( {\frac{1}{f} - \frac{1}{{\left( {M_{0} + 1} \right)f} - \frac{D_{0}}{N_{0}} + \frac{D_{ML}D}{\Delta}}} \right)^{- 1}}}} & (8)\end{matrix}$

That is, the distance between the object 100 and the imaging lens 10 canbe determined from the image shift length Δ.

In a case where the microlens array 20 is not provided, and the imagingelement 30 is located on the virtual imaging surface 70, an image of theobject is formed on the virtual imaging surface 70. In a case wherelight rays emitted from the same object are divided by microlenses, andimages are formed on the surface of the imaging element 30 located infront of the virtual imaging surface 70 as in this embodiment, the sameobject is imaged more than once in accordance with parallaxes.Accordingly, images formed by the microlenses imaging the same objectmore than once (microlens images) are output as an image.

The microlens images are images formed when the microlenses are arrangedat regular pitch. The microlens images are reduced from the image, whichshould have been formed on the virtual imaging surface 70, by themagnification N (according to the equation (4)) of the microlens imagingsystem. The obtained microlens images are re-formed as a two-dimensionalimage without an overlapping portion, by subjecting the read image dataof each microlens to a re-forming operation. The re-forming operationwill be described later.

Having parallaxes smaller than the aperture of the imaging lens 10, theobtained microlens images can be subjected to a three-dimensionalprocessing operation that uses parallaxes.

FIGS. 2 and 3 show plane arrangement methods that can be used for themicrolens array 20 in this embodiment. FIG. 2 is a diagram showing anexample case where the microlenses 21 of the microlens array 20 arearranged in a hexagonal closest packed array, and FIG. 3 is a diagramshowing an example case where the microlenses 21 of the microlens array20 are squarely arranged. The distance between the centers 211 of themicrolenses 21 is represented by D_(ml).

Here, the microlens array 20 is placed on the image entering side of theimaging element 30. When the components are stacked on a plane surface,an angle θ is the angle between the X-axis representing one direction inwhich the pixels 31 are aligned, and the X′-axis of the microlens array20 defined by a straight line connecting the centers 211 of microlenses21.

Referring now to FIG. 4, a method of re-forming a two-dimensional imagewithout any overlapping portion from microlens images formed byrepeatedly imaging the same object is described.

FIG. 4 shows an example case where first through third microlenses areadjacent to one another, and form microlens images 91 a, 91 b, and 91 c,respectively, on the surface of the imaging element 30. The visualfields in which the microlens images 91 a, 91 b, and 91 c are formed arevisual fields 93 a, 93 b, and 93 c that overlap with one another on thevirtual imaging surface 70. FIG. 4 illustrates a case where thereduction rate N is 0.5. As a result of enlarging each visual field by0.5, each object point is imaged two or more times. When the reductionratio N is 0.5, the image on the virtual imaging surface 70 can bereproduced by enlarging each microlens image by 1/N (=2, when N=0.5). InFIG. 4, reference numeral 92 a indicates the image center of the firstmicrolens, and reference numeral 94 a indicates the center of the visualfield of the first microlens on the virtual imaging surface 70.

In this embodiment, overlapping portions of the visual fields areprocessed by subjecting the luminance values output from the pixels toan averaging operation. In this manner, one image can be re-formed fromdata that has overlapping portions.

FIG. 5 shows overlaps in the image data of the respective microlenses atthe time of image re-formation. The seven circles represent the imageareas subjected to an enlarging operation that is performed mainly onthe center of each microlens image. For example, shaded portions 96 aand 96 b are the overlapping portions of the microlens image data inthis case. At the overlapping portions, the corresponding two or threesets of image data is subjected to averaging, to form a single image. Inthe process of averaging, the random noise in each microlens image iseffectively reduced.

Normally, when random noises having no correlation with one another aresubjected to averaging N times, the standard deviation of noise becomes1/N^(1/2). Therefore, as the number N of overlaps becomes larger, thereduction in image noise becomes larger.

However, when random noises having a correlation with one another aresubjected to averaging, noises are not reduced. For example, if thereare horizontal linear noises in FIG. 5, the added microlens images havecommon horizontal linear noises at the added portions 96 a and 96 b thatare the shaded portions in the drawing. Therefore, a noise reductioneffect cannot be expected.

In an image element, pixels are normally selected in order of rownumber, and columns are normally read in parallel. Therefore, horizontallinear noises or vertical linear noises are liable to appear.

In a case where the added microlens images have common horizontal orvertical linear noises, a straight line connecting the centers ofmicrolenses coincides with the X-axis direction or the Y-axis directionformed by the pixel array of the imaging element 30. Referring now toFIG. 6, the condition for the axis directions to match is described.

FIG. 6 shows three microlenses 211, 212, and 213. The grids representthe pixel array. Here, the straight line connecting the centers of themicrolenses forms an angle θ with the horizontal axis of the pixels. Thecenter-to-center distance between the microlenses (the microlens pitch)is represented by D_(ml), and the pixel pitch is represented by dp. Ifthe pixel pitch in the X-direction and the pixel pitch in theY-direction differ from each other, the pitch in the Y-direction isrepresented by dp. For example, pixels 221 and 222 exist in themicrolenses 211 and 212, and are located on the straight line connectingthe centers of the microlenses 211 and 212. If the distance between thepixels 221 and 222 is ½×D_(ml), the microlens images at these pixelsoverlap with each other and are added when each of the microlens imagesis enlarged by 2 (=1/N). The condition for the pixel 221 and the pixel222 to be located in different rows from each other at this point isexpressed as follows.½×D _(ml)×sin θ>dp  (9)

If D_(ml)/dp is 20, for example, sin θ should be greater than 0.1.Accordingly, when the angle θ is not smaller than −sin⁻¹(0.1) and notlarger than sin⁻¹(0.1), the pixels might not overlap with each other.The reason that the distance between the pixels is assumed to be½×D_(ml) is that pixels located at a distance of ½×D_(ml) or longer fromeach other often overlap with each other.

Referring now to FIGS. 7 and 8, angles θ that should be avoided aredescribed. FIG. 7 shows a microlens array that is a hexagonal closestpacked array. First, straight lines connecting the centers ofmicrolenses are described. Where the direction in which microlenses arealigned at the highest density is the direction at 0°, the abovedescribed noise reduction effect cannot be expected when the axis in thehorizontal direction of the pixel array coincides with the direction at0°. The same applies in cases where the angle θ is 30° and 60°, as shownin FIG. 7. For example, in cases where the 0° axis of the microlensarray forms an angle of 15° with the horizontal axis of the pixels,pixels in different rows overlap with each other and are added.Therefore, such cases are preferable.

As can be seen from FIG. 8, in a square array, the above mentioned noisereduction effect cannot be expected when the angle θ is 0°, 26°, or 45°.

FIG. 9 shows the angles θ that should be avoided in a hexagonal closestpacked array. In FIG. 9, angles are determined, with dp/D_(ml) beingequal to 1/20. The error bars indicate the conditions for two pixelslocated at a distance of ½×D_(ml) from each other to exist in the samerow. When the target microlens is to overlap with a microlens that islocated one microlens outside, the error bars indicate the ranges ofangles θ expressed as follows according to the equation (9).½×D _(ml)×sin θ≦dp

That is, in the case of a hexagonal closest packed array, the angles θto be avoided are expressed as follows.−sin⁻¹(2dp/D _(ml))≦θ≦sin⁻¹(2dp/D _(ml))30°−sin⁻¹(2dp/D _(ml))≦θ≦30°+sin⁻¹(2dp/D _(ml))60°−sin⁻¹(2dp/D _(ml))≦θ≦60°+sin⁻¹(2dp/D _(ml))

FIG. 10 shows the angles θ to be avoided in a case where the microlensesare squarely arranged. In FIG. 10, angles are determined, with dp/D_(ml)being equal to 20. In the case of a square array, the angles θ to beavoided are expressed as follows.−sin⁻¹(2dp/D _(ml))≦θ≦sin⁻¹(2dp/D _(ml))26°−sin⁻¹(2dp/D _(ml))≦θ≦26°+sin⁻¹(2dp/D _(ml))45°−sin⁻¹(2dp/D _(ml))≦θ≦45°+sin⁻¹(2dp/D _(ml))

As described above, the microlenses are arranged so that the axis formedby a straight line connecting the centers of microlenses forms aconstant angle with the horizontal axis of pixels. In this manner, a lownoise solid-state imaging device can be realized.

As described so far, this embodiment can provide a solid-state imagingdevice that can restore the S/N ratio and the resolution of an imageeven when the image is re-formed.

This embodiment can also provide a solid-state imaging device that canobtain distance information and improve SNR, without false colorgeneration at the time of imaging of an object at high spatialfrequency.

As shown in FIG. 11, any of the above described solid-state imagingdevices may include a drive unit 300 that drives the imaging element 30,and a processing unit 320 that processes signals output from the imagingelement 30. Any of the above described solid-state imaging devices mayalso include a power supply 330 that is necessary for driving theimaging element 30. Any of the above described solid-state imagingdevices may also include an output unit 340 such as a display. An outputdevice (not shown) that is provided outside may be connected to thesolid-state imaging device. Each signal output from the imaging element30 is displayed by the output unit 340 or the output device.

Any of the above described solid-state imaging devices may be a mobilecommunication terminal, a digital camera, a personal computer, or asurveillance camera, for example.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fail within thescope and spirit of the inventions.

What is claimed is:
 1. A solid-state imaging device comprising: animaging element including an imaging area formed with a plurality ofpixel blocks each including a plurality of pixels; a first opticalsystem configured to form an image of an object on an imaging surface;and a second optical system configured to re-form the image, which hasbeen formed on the imaging surface, on the plurality of pixel blockscorresponding to a plurality of microlenses, the second optical systemincluding a microlens array formed with the plurality of microlensesprovided in accordance with the plurality of pixel blocks, wherein theplurality of microlenses are arranged in such a manner that an angle θbetween a straight line connecting center points of adjacent microlensesand one of a row direction and a column direction in which the pixelsare aligned is expressed as follows:θ>sin⁻¹(2dp/D _(ml)) where D_(ml) represents microlens pitch, and dprepresents pixel pitch.
 2. The device according to claim 1, wherein theplurality of microlenses are arranged in a hexagonal closest packedarray, and the angle θ is not included in the following ranges:30°−sin⁻¹(2dp/D _(ml))≦θ≦30°+sin⁻¹(2dp/D _(ml))60°−sin⁻¹(2dp/D _(ml))≦θ≦60°+sin⁻¹(2dp/D _(ml)).
 3. The device accordingto claim 1, wherein the plurality of microlenses are arranged in asquare array, and the angle θ is not included in the following ranges:26°−sin⁻¹(2dp/D _(ml))≦θ≦26°+sin⁻¹(2dp/D _(ml))45°−sin⁻¹(2dp/D _(ml))≦θ≦45°+sin⁻¹(2dp/D _(ml)).
 4. The device accordingto claim 1, further comprising a processing unit configured to process asignal output from the imaging element.
 5. The device according to claim1, further comprising a drive unit configured to drive the imagingelement.
 6. The device according to claim 1, further comprising a powersupply connected to the imaging element.